USHIFT-7223: Automate Polarion tests OCP-66820 and OCP-66882 by agullon · Pull Request #6905 · openshift/microshift

agullon · 2026-06-18T14:39:05Z

Summary

Automate two Polarion test cases ported from openshift-tests-private:

OCP-66820: Version file 3 minors behind causes prerun failure logged to prerun_failed.log and reported by greenboot
OCP-66882: Removing /var/lib/microshift with healthy health.json causes MicroShift to start fresh

Tests are added to el98-src@backup-and-restore-on-reboot (presubmits) and el98-lrel@backups (releases).

Greenboot handling

Both tests can trigger greenboot rollback loops (intentional prerun failure, slow fresh bootstrap on ARM). A Reboot MicroShift Host And Wait For Greenboot keyword waits up to 10m for greenboot to settle and verifies system stability by comparing boot IDs before and after a delay to detect pending rollback reboots.

Test randomization is disabled in both scenarios because the prerun tests are destructive (data deletion, greenboot rollback loops) and must run last to avoid destabilizing subsequent test suites.

Test plan

el98-src@backup-and-restore-on-reboot passes on x86 and ARM
el98-lrel@backups passes on x86 and ARM
Existing backup tests in both scenarios are unaffected

openshift-ci-robot · 2026-06-18T14:39:09Z

@agullon: This pull request references USHIFT-7223 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Summary

Add Robot Framework tests for two prerun data management scenarios in test/suites/backup/prerun-data-management.robot

OCP-66820: Verifies that modifying the version file to be 3 minors back causes MicroShift to fail and log the failure reason to prerun_failed.log, reported by greenboot healthcheck

OCP-66882: Verifies that removing /var/lib/microshift while health.json shows healthy causes MicroShift to start fresh, ignoring stale health info

Both tests added to existing CI scenarios (el98-src@backup-and-restore-on-reboot.sh and el98-lrel@backups.sh)

Test plan

Run el98-src@backup-and-restore-on-reboot presubmit scenario — verify both existing and new tests pass

Run el98-lrel@backups release scenario — verify both existing and new tests pass

Verify OCP-66820 test correctly detects version mismatch failure in prerun_failed.log

Verify OCP-66882 test correctly detects fresh start journal messages after data removal

🤖 Generated with Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

coderabbitai · 2026-06-18T14:39:29Z

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

@coderabbitai resume to resume automatic reviews.
@coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

▶️ Resume reviews
🔍 Trigger review

Walkthrough

A new Robot Framework suite prerun-data-management.robot is added under test/suites/backup/. It covers two end-to-end scenarios: prerun failure when the version file is too old, and fresh-start behavior when data is missing but persisted health is healthy. Two existing scenario scripts are updated to include this suite in their run_tests invocations.

Changes

Prerun Data Management Tests

Layer / File(s)	Summary
New prerun-data-management Robot suite `test/suites/backup/prerun-data-management.robot`	Adds a 177-line Robot suite with settings, path variables, two test cases (version-too-old prerun failure and data-missing fresh-start), and keywords for version file manipulation via `jq`, greenboot service polling, journal/log assertions, directory cleanup, reboot, and MicroShift service lifecycle.
Scenario script wiring `test/scenarios/presubmits/el98-src@backup-and-restore-on-reboot.sh`, `test/scenarios/releases/el98-lrel@backups.sh`	Both `scenario_run_tests` functions updated to pass `prerun-data-management.robot` as an additional suite to `run_tests host1`, reformatted as multi-line invocations.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Suggested reviewers

pmtk
pacevedom

🚥 Pre-merge checks | ✅ 15

✅ Passed checks (15 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the main change: automating the two Polarion tests added in this PR.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	The added test cases use fixed, descriptive Robot names and contain no dynamic values; no unstable Ginkgo-style titles were introduced.
Test Structure And Quality	✅ Passed	Not applicable: the PR adds Robot Framework tests, not Ginkgo. The suite uses setup/teardown, explicit waits, and per-test cleanup.
Microshift Test Compatibility	✅ Passed	New Robot tests only use MicroShift host/system commands and allowed health checks; no forbidden OpenShift APIs or HA assumptions found.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	PR adds Robot suite and scenario wiring only; no new Ginkgo e2e tests are present, so the SNO check is not applicable.
Topology-Aware Scheduling Compatibility	✅ Passed	Only test/scenario files changed; no manifests, controllers, replicas, affinity, nodeSelectors, or PDBs were introduced.
Ote Binary Stdout Contract	✅ Passed	The patch only changes shell scenario wrappers and a Robot suite; no main/TestMain/BeforeSuite-style code or stdout-writing APIs were introduced.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	PASS — The new Robot tests use only SSH/local system commands and file checks; no IPv4-only parsing or public-internet connectivity found.
No-Weak-Crypto	✅ Passed	Touched files only add Robot/bash test plumbing; no MD5/SHA1/DES/RC4/3DES/Blowfish, ECB, custom crypto, or secret comparisons found.
Container-Privileges	✅ Passed	No added privileged/hostPID/hostNetwork/hostIPC/SYS_ADMIN/allowPrivilegeEscalation/root settings appear in the PR diff; OVN security fields were unchanged.
No-Sensitive-Data-In-Logs	✅ Passed	Added log assertions only for version-mismatch and fresh-start messages; no passwords, tokens, PII, or host/customer data are logged in the changed files.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

agullon · 2026-06-19T08:39:04Z

/retest

agullon · 2026-06-22T06:42:14Z

/retest

agullon · 2026-06-23T14:35:27Z

/retest

agullon · 2026-06-24T05:35:11Z

/retest

ARM (Graviton) CI instances take significantly longer to complete reboot cycles compared to x86. SSH typically becomes available in ~60-120s on ARM vs ~30-50s on x86. When tests involve destructive operations like removing /var/lib/microshift followed by a fresh bootstrap (certificate generation, etcd initialization), the total reboot cycle on ARM can reach ~400s, exceeding the previous 5m (300s) timeout. This was observed in PR openshift#6905 where the new prerun-data-management tests triggered reboot timeouts exclusively on ARM jobs, while identical x86 runs passed comfortably. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> pre-commit.check-secrets: ENABLED

agullon · 2026-06-24T10:25:02Z

/test e2e-aws-tests-release
/test e2e-aws-tests-release-arm

agullon · 2026-06-24T13:45:42Z

/label tide/merge-method-squash

agullon · 2026-06-24T18:49:02Z

/test e2e-aws-tests-release
/test e2e-aws-tests-release-arm
/test e2e-aws-tests
/test e2e-aws-tests-arm

Add Robot Framework tests for two prerun data management scenarios ported from openshift-tests-private: - OCP-66820: verify that a version file 3 minors behind the running executable causes MicroShift to fail on reboot, with the failure reason logged to prerun_failed.log and reported by greenboot. - OCP-66882: verify that removing /var/lib/microshift while health.json shows healthy causes MicroShift to start fresh, ignoring stale health info. Both tests use a Reboot MicroShift Host And Wait For Greenboot keyword that tolerates greenboot rollback loops: it waits up to 10 minutes for greenboot to settle and verifies system stability by comparing boot IDs before and after a delay to detect pending rollback reboots. Test randomization is disabled in both backup scenarios because prerun-data-management.robot is destructive (deletes MicroShift data, triggers greenboot rollback loops) and must run last. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> pre-commit.check-secrets: ENABLED

agullon · 2026-06-24T21:12:15Z

/test e2e-aws-tests-release
/test e2e-aws-tests-release-arm
/test e2e-aws-tests
/test e2e-aws-tests-arm

Run "Data Missing" before "Version Too Old" so the healthy reboot hits a fresh system. "Version Too Old" triggers greenboot rollback loops that degrade ARM I/O performance, causing subsequent healthy reboots to exceed the 5-minute timeout. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> pre-commit.check-secrets: ENABLED

agullon · 2026-06-25T06:22:03Z

/test e2e-aws-tests-release
/test e2e-aws-tests-release-arm
/test e2e-aws-tests
/test e2e-aws-tests-arm

agullon · 2026-06-25T10:50:27Z

/override ci/prow/e2e-aws-tests-arm
/override ci/prow/e2e-aws-tests-bootc-release-el10

openshift-ci · 2026-06-25T10:50:58Z

@agullon: Overrode contexts on behalf of agullon: ci/prow/e2e-aws-tests-arm, ci/prow/e2e-aws-tests-bootc-release-el10

Details

In response to this:

/override ci/prow/e2e-aws-tests-arm
/override ci/prow/e2e-aws-tests-bootc-release-el10

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci · 2026-06-25T10:51:02Z

@agullon: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

agullon · 2026-06-25T10:54:08Z

/verified by CI

openshift-ci-robot · 2026-06-25T10:54:21Z

@agullon: This PR has been marked as verified by CI.

Details

In response to this:

/verified by CI

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

kasturinarra · 2026-06-25T17:26:52Z

/lgtm

openshift-ci · 2026-06-25T17:31:44Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: agullon, kasturinarra

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [agullon,kasturinarra]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jun 18, 2026

openshift-ci Bot requested review from ggiguash and jogeo June 18, 2026 14:40

openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 18, 2026

coderabbitai Bot approved these changes Jun 18, 2026

View reviewed changes

agullon force-pushed the USHIFT-7223 branch from 5a55469 to 0f95473 Compare June 24, 2026 13:45

openshift-ci Bot added the tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges. label Jun 24, 2026

agullon marked this pull request as draft June 24, 2026 15:57

openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 24, 2026

agullon force-pushed the USHIFT-7223 branch from 01f88f1 to ab2baa8 Compare June 24, 2026 21:11

agullon marked this pull request as ready for review June 24, 2026 21:11

openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 24, 2026

openshift-ci Bot requested a review from jerpeter1 June 24, 2026 21:13

agullon marked this pull request as draft June 25, 2026 06:18

openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 25, 2026

agullon marked this pull request as ready for review June 25, 2026 08:15

openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 25, 2026

openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Jun 25, 2026

openshift-ci Bot assigned kasturinarra Jun 25, 2026

openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jun 25, 2026

openshift-merge-bot Bot merged commit c3b0b39 into openshift:main Jun 25, 2026
19 checks passed

Uh oh!

Conversation

agullon commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Greenboot handling

Test plan

Uh oh!

openshift-ci-robot commented Jun 18, 2026 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

coderabbitai Bot commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews paused

Walkthrough

Changes

Estimated code review effort

Suggested reviewers

Uh oh!

agullon commented Jun 19, 2026

Uh oh!

agullon commented Jun 22, 2026

Uh oh!

agullon commented Jun 23, 2026

Uh oh!

agullon commented Jun 24, 2026

Uh oh!

agullon commented Jun 24, 2026

Uh oh!

agullon commented Jun 24, 2026

Uh oh!

agullon commented Jun 24, 2026

Uh oh!

agullon commented Jun 24, 2026

Uh oh!

agullon commented Jun 25, 2026

Uh oh!

agullon commented Jun 25, 2026

Uh oh!

openshift-ci Bot commented Jun 25, 2026

Uh oh!

openshift-ci Bot commented Jun 25, 2026

Uh oh!

agullon commented Jun 25, 2026

Uh oh!

openshift-ci-robot commented Jun 25, 2026

Uh oh!

kasturinarra commented Jun 25, 2026

Uh oh!

openshift-ci Bot commented Jun 25, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

agullon commented Jun 18, 2026 •

edited

Loading

openshift-ci-robot commented Jun 18, 2026 •

edited by openshift-ci Bot

Loading

coderabbitai Bot commented Jun 18, 2026 •

edited

Loading